Meaningful patient and public involvement in digital health innovation, implementation and evaluation: A systematic review.

INTRODUCTION: The importance of meaningfully involving patients and the public in digital health innovation is widely acknowledged, but often poorly understood. This review, therefore, sought to explore how patients and the public are involved in digital health innovation and to identify factors that support and inhibit meaningful patient and public involvement (PPI) in digital health innovation, implementation and evaluation. METHODS: Searches were undertaken from 2010 to July 2020 in the electronic databases MEDLINE, EMBASE, PsycINFO, CINAHL, Scopus and ACM Digital Library. Grey literature searches were also undertaken using the Patient Experience Library database and Google Scholar. RESULTS: Of the 10,540 articles identified, 433 were included. The majority of included articles were published in the United States, United Kingdom, Canada and Australia, with representation from 42 countries highlighting the international relevance of PPI in digital health. 112 topic areas where PPI had reportedly taken place were identified. Areas most often described included cancer (n = 50), mental health (n = 43), diabetes (n = 26) and long-term conditions (n = 19). Interestingly, over 133 terms were used to describe PPI; few were explicitly defined. Patients were often most involved in the final, passive stages of an innovation journey, for example, usability testing, where the ability to proactively influence change was severely limited. Common barriers to achieving meaningful PPI included data privacy and security concerns, not involving patients early enough and lack of trust. Suggested enablers were often designed to counteract such challenges. CONCLUSIONS: PPI is largely viewed as valuable and essential in digital health innovation, but rarely practised. Several barriers exist for both innovators and patients, which currently limits the quality, frequency and duration of PPI in digital health innovation, although improvements have been made in the past decade. Some reported barriers and enablers such as the importance of data privacy and security appear to be unique to PPI in digital innovation. Greater efforts should be made to support innovators and patients to become meaningfully involved in digital health innovations from the outset, given its reported benefits and impacts. Stakeholder consensus on the principles that underpin meaningful PPI in digital health innovation would be helpful in providing evidence-based guidance on how to achieve this. PATIENT OR PUBLIC CONTRIBUTION: This review has received extensive patient and public contributions with a representative from the Patient Experience Library involved throughout the review's conception, from design (including suggested revisions to the search strategy) through to article production and dissemination. Other areas of patient and public contributor involvement include contributing to the inductive thematic analysis process, refining the thematic framework and finalizing theme wording, helping to ensure relevance, value and meaning from a patient perspective. Findings from this review have also been presented to a variety of stakeholders including patients, patient advocates and clinicians through a series of focus groups and webinars. Given their extensive involvement, the representative from the Patient Experience Library is rightly included as an author of this review.

The Potential Impacts of a Digital Preoperative Assessment Service on Appointments, Travel-Related Carbon Dioxide Emissions, and User Experience: Case Study.

BACKGROUND: The National Health Service (NHS) cannot keep up with the demand for operations and procedures. Preoperative assessments can be conducted on the internet to improve efficiency and reduce wait times for operations. MyPreOp is a cloud-based platform where patients can complete preoperative questionnaires. These are reviewed by a nurse who determines whether they need a subsequent face-to-face appointment. OBJECTIVE: The primary objective of this study is to describe the potential impact of MyPreOp (Ultramed Ltd) on the number of face-to-face appointments. The secondary objectives are to examine the time spent on preoperative assessments completed using MyPreOp in NHS Trusts and user ratings of usability and acceptability. METHODS: The study design was a case study service evaluation. Data were collected using the MyPreOp system from 2 NHS Trusts (Guy's and St Thomas' and Royal United Hospitals Bath) and the private BMI Bath Clinic during the 4-month period from September to December 2020. Participants were adults of any age and health status at the participating hospitals who used MyPreOp to complete a preoperative assessment before a scheduled surgery. The primary outcome was the number of face-to-face appointments avoided by patients who used MyPreOp. The investigated secondary outcomes included the length of time spent by nurses completing preoperative assessments, associated travel-related carbon dioxide emissions compared with standard care, and quantitative user feedback. User feedback was assessed at all 3 sites; however, the other outcomes could only be examined in the Royal United Hospitals Bath sample because of data limitations. RESULTS: Data from 2500 participants were included. Half of the assessed patients did not need a further face-to-face appointment and required a median of only 5.3 minutes of nurses' time to review. The reduction in appointments was associated with a small saving of carbon dioxide equivalent emissions (9.05 tons). Patient feedback was generally positive: 79.8% (317/397) of respondents rated MyPreOp as easy or very easy to use, and 85.2% (340/399) thought the overall experience was good or very good. CONCLUSIONS: This evaluation demonstrates the potential benefits of MyPreOp. However, further research using rigorous scientific methodology and a larger sample of NHS Trusts and users is needed to provide strong evidence of MyPreOp's efficacy, usability, and cost-effectiveness.

Methods of usability testing in the development of eHealth applications: A scoping review.

BACKGROUND: The number of eHealth applications has exponentially increased in recent years, with over 325,000 health apps now available on all major app stores. This is in addition to other eHealth applications available on other platforms such as PC software, web sites and even gaming consoles. As with other digital applications, usability is one of the key factors in the successful implementation of eHealth apps. Reviews of the literature on empirical methods of usability testing in eHealth were last published in 2015. In the context of an exponentially increasing rate of App development year on year, an updated review is warranted. OBJECTIVE: To identify, explore, and summarize the current methods used in the usability testing of eHealth applications. METHODS: A scoping review was conducted on literature available from April 2014 up to October 2017. Four databases were searched. Literature was considered for inclusion if it was (1) focused on an eHealth application (which includes websites, PC software, smartphone and tablet applications), (2) provided information about usability of the application, (3) provided empirical results of the usability testing, (4) a full or short paper (not an abstract) published in English after March 2014. We then extracted data pertaining to the usability evaluation processes described in the selected studies. RESULTS: 133 articles met the inclusion criteria. The methods used for usability testing, in decreasing order of frequency were: questionnaires (n = 105), task completion (n = 57), 'Think-Aloud' (n = 45), interviews (n = 37), heuristic testing (n = 18) and focus groups (n = 13). Majority of the studies used one (n = 45) or two (n = 46) methods of testing. The rest used a combination of three (n = 30) or four (n = 12) methods of testing usability. None of the studies used automated mechanisms to test usability. The System Usability Scale (SUS) was the most frequently used questionnaire (n = 44). The ten most frequent health conditions or diseases where eHealth apps were being evaluated for usability were the following: mental health (n = 12), cancer (n = 10), nutrition (n = 10), child health (n = 9), diabetes (n = 9), telemedicine (n = 8), cardiovascular disease (n = 6), HIV (n = 4), health information systems (n = 4) and smoking (n = 4). Further iterations of the app were reported in a minority of the studies (n = 41). The use of the 'Think-Aloud' (Pearson Chi-squared test: χ2 = 11.15, p < 0.05) and heuristic walkthrough (Pearson Chi-squared test: χ2 = 4.48, p < 0.05) were significantly associated with at least one further iteration of the app being developed. CONCLUSION: Although there has been an exponential increase in the number of eHealth apps, the number of studies that have been published that report the results of usability testing on these apps has not increased at an equivalent rate. The number of digital health applications that publish their usability evaluation results remains only a small fraction. Questionnaires are the most prevalent method of evaluating usability in eHealth applications, which provide an overall measure of usability but do not pinpoint the problems that need to be addressed. Qualitative methods may be more useful in this regard. The use of multiple evaluation methods has increased. Automated methods such as eye tracking have not gained traction in evaluating health apps. Further research is needed into which methods are best suited for the different types of eHealth applications, according to their target users and the health conditions being addressed.